E155 Final Project - Invisible Drumset
  • Home
  • Project

Project Documentation

Abstract

This project aims to adapt the existing Invisible Drum Set—originally implemented with an Arduino Nano and BNO055 IMU sensor—onto the E155 hardware platform, consisting of an STM32 MCU and UPduino FPGA. It will enable playing virtual drums through air gestures detected by the IMU, with real-time audio synthesis and low latency.

System Overview

The data pipeline in our system relies on multiple communication protocols between the different hardware components. The BNO085 sensor communicates with the Arduino Nano ESP32 using the I²C protocol, with the high-level SHTP protocol layered on top to enable structured transfer of fused motion data. Once the sensor data is received by the ESP32, it is forwarded to the FPGA over an SPI interface, where the FPGA operates as the SPI slave and the ESP32 serves as the SPI master. On the FPGA, the data is processed and packaged into the custom transmission format used for the system.

From there, the FPGA sends the packaged data to the STM32 microcontroller through a second SPI link, again with the FPGA acting as the slave and the STM32 as the master. In the microcontroller, the data is further interpreted to identify which drum corresponds to the detected motion input. The resulting drum-selection signal is then output through a digital-to-analog converter (DAC) to drive a speaker that plays the correct drum sound.

This multi-protocol architecture enables reliable motion data transfer across all computing elements in the system.

Data Flow: BNO085 Sensor (I2C/SHTP) → Arduino ESP32 → [SPI] → FPGA → [SPI] → STM32 MCU → DAC → Speaker

Hardware Components

BNO085 IMU Sensor

The BNO085 is a 9-axis intelligent IMU that integrates an accelerometer, gyroscope, and magnetometer with an onboard sensor-fusion processor. It communicates through the Sensor Hub Transport Protocol (SHTP), a structured, packet-based protocol that runs over I²C, SPI, or UART.

Key Features: - Onboard ARM Cortex-M0+ microcontroller running Bosch’s BSX Lite fusion firmware - Performs sensor fusion (combining accel + gyro + mag) internally - Provides high-level motion data like quaternions and rotation vectors - SHTP organizes communication into channels with sequence numbers for reliable packet delivery

Arduino Nano ESP32

The Arduino Nano ESP32 combines the ESP32-S3’s processing power with the Arduino ecosystem, providing integrated Wi-Fi and Bluetooth connectivity. The system uses the Adafruit_BNO08x library, which abstracts low-level SHTP packet parsing and directly exposes orientation and motion data. Communication with the BNO085 uses I²C protocol with default address (0x4A).

Library: Adafruit_BNO08x

FPGA (UPduino)

The FPGA serves as a data bridge between the Arduino ESP32 and STM32 MCU: - Receives sensor data from ESP32 via SPI (slave mode) - Processes and packages data into custom transmission format - Sends packaged data to STM32 MCU via SPI (slave mode) - Implements dual SPI slave interfaces

Key Modules: drum_trigger_top.sv, arduino_spi_slave.sv, spi_slave_mcu.sv

STM32 MCU

The STM32L432KC microcontroller receives gesture data from the FPGA via SPI, unpacks WAV files from flash memory (parsing RIFF headers and extracting audio samples), and plays the samples through the 12-bit DAC on PA4 at the correct sample rate to drive the speaker.

Results

SPI Data Transmission: ESP32 → FPGA

Validation using logic analyzer confirms correct data transfer. The packet begins with header byte 0xAA, followed by three 16-bit signed values for Euler angles (roll, pitch, yaw).

Parameter Raw Bytes (Hex) Value (Decimal) Interpretation
Header AA — Packet start marker
Roll 03 74 0x0374 = 884 ~88.4° (divide by 10)
Pitch FF 31 0xFF31 = –207 ~–20.7° (divide by 10)
Yaw 1F 0D 0x1F0D = 7949 ~794.9° (before wrap-around)

The decoded values match Arduino serial monitor output, confirming SPI protocol is functioning correctly and data formatting matches the intended system design.

SPI Data Transmission: FPGA → MCU

SPI communication between FPGA and MCU was validated using a logic analyzer, confirming correct data transfer and protocol compliance.

DAC Output

The 12-bit DAC on PA4 successfully outputs audio samples at the correct sample rate, enabling real-time drum sound playback through the speaker.

Schematics & Diagrams

TODO: Add complete system block diagram and pin connection diagram

Please add: - System block diagram showing all components and communication protocols - Pin connection diagram with all interconnections, signal names, and pin numbers

Save diagrams to Images/ directory.

Code Repository

TODO: Add GitHub repository URL

View Complete Source Code on GitHub

The repository contains all source code, including FPGA Verilog modules, MCU C code, Arduino/ESP32 code, testbenches, and documentation files.

Photos & Video

TODO: Add photos and video

Please add: - Photos of the complete system, hardware components, and wiring - Video demonstration showing the system in operation

Video should be hosted in the provided Google Drive folder and named: <LastName1><LastName2><VideoName> (e.g., BellidoStralkaInvisibleDrumSet)

Bill of Materials

Material Quantity Cost Total
BNO085 IMU Sensor 2 $24.95 $49.90
Speaker (8Ω, 4W) 1 Stockroom $0.00
Switches (drums) 2 Stockroom $0.00
Potentiometer 1 Stockroom $0.00
SD Card 1 Stockroom $0.00
TOTAL $49.90

Total estimated cost under $50, excluding development hardware.

References

  • BNO085 Datasheet
  • BNO085 Sensor (Adafruit)
  • STM32L432KC Documentation
  • UPduino Documentation
  • Invisible Drum Set (Original)
  • HMC E155 Project Website
  • Arduino ESP32 Spec
  • Adafruit_BNO08x Library
  • HMC E155 Lab 7 SPI Code

Acknowledgements

We want to thank staff engineer Xavier Walter and Professor Matthew Spencer for their constant support.